Commands sent to a server may be implemented as remote procedure calls. A remote procedure call allows a client to invoke any simple or complex operation on the server by means of a simple procedure call abstraction. An RPC subsystem generates client server stubs to send and receive parameters, freeing the application from this complexity.
Many client-server systems implement each invocation of a command as a separate "round trip" to the server. In such a scenario, each command invocation is delayed due to the network latency, the delay (approximately 4 milliseconds in many networks) that occurs when a client command is sent to the server. In addition, execution of each command requires the server to perform context switches, requiring resources on the server for each such command it receives.
Another problem with single remote procedure calls is that they use system resources extremely inefficiently. Basic system resources include the client, the network and the server. When a client prepares a single RPC, the server and the network connection between the client and server sit idle. When the client sends the RPC over the network, the client and server sit idle. When the server processes the RPC, the client and the network connection between the client and server sit idle. When the response is transmitted to the client over the network, the server and client sit idle. As a result of this inefficiency, the throughput of the system, that is, the number of requests for service processed in a given amount of time, is low.
To amortize the network latency and context switching over several commands, multiple commands may be bundled together. However, this approach does not achieve optimal efficiency of the system. The client still sits idle during transmission of the bundle and its result and during server processing of the bundle. The server and network are still idle while the client assembles the bundle.
Although the efficiency, and therefore, throughput, of a bundled RPC system improves over that of a single-RPC-at-a-time, response time is actually worse, because the first RPC to arrive in the bundle must wait until additional RPCs arrive to produce the bundle. Conventional systems which perform bundling in the client using a process external to the application are especially prone to this problem, because the application had no control over the process. If the client has a high priority RPC, it would be placed by the external process into a queue of the external process waiting for enough RPCs to arrive to produce a bundle before the high-priority RPC would be sent.
It would be possible to allow the client bundling to be performed by the application to give the application more control over the bundling process, allowing high priority commands to be sent right away and low priority commands to be bundled. However, such an approach causes other problems. If the application sends too many small bundles, the overhead associated with calling the RPC system too often will cause its own efficiency problems. If the application sends too few large bundles, the response time problem above is made worse. In addition, the application is made more complex because the application is required to perform its own memory management functions. Another problem with this approach is that it would likely increase the memory requirements in the client because not only would the application require a buffer for storage of the bundle, but when the bundle is provided to a transport process, the transport process will require its own buffer for reliable transmission of the bundle.
Whether the bundling is performed by the application or an external process, the error handling required to recover from errors becomes extremely complex because there may be outstanding RPCs at the time the error occurs and correct handling of any error may depend on which command caused the error.
Therefore, a system and method is desirable that improves the efficiency and throughput of the system by reducing or even eliminating idle time of the client, network connection and server, amortizes network latency, context switching and RPC subsystem overhead without imposing significant delay, does not require the application to perform complex memory management functions nor requires two buffers to queue and reliably send the bundles, and simplifies programming of error handling.